# Visual Model

FastVLM is an efficient visual encoding model designed specifically for visual language models. It uses the innovative FastViTHD hybrid visual encoder to reduce the time required for encoding high-resolution images and the number of output tokens, resulting in excellent performance in both speed and accuracy. FastVLM is primarily positioned to provide developers with powerful visual language processing capabilities, applicable to various scenarios, particularly performing excellently on mobile devices that require rapid response.

Document Inlining is a composite AI system launched by Fireworks AI that transforms any large language model (LLM) into a visual model to handle images or PDF documents. This technology utilizes automated processes to convert any digital asset format into an LLM-compatible format, enabling logical reasoning. Document Inlining parses images and PDFs directly into the chosen LLM, offering improved quality, input flexibility, and an exceptionally simple user experience. It addresses the limitations of traditional LLMs when handling non-text data by breaking tasks down into specialized components, enhancing the quality of textual model reasoning while simplifying the developer experience.

InternViT-6B-448px-V2_5 is a visual model built upon InternViT-6B-448px-V1-5, which enhances the visual encoder's ability to extract visual features by utilizing ViT incremental learning and NTP loss (Stage 1.5). It particularly excels in domains where representation is lacking in large-scale network datasets, such as multilingual OCR data and mathematical charts. This model is part of the InternVL 2.5 series, maintaining the same 'ViT-MLP-LLM' architecture as its predecessor, while integrating the newly incrementally pretrained InternViT alongside various pretrained LLMs, including InternLM 2.5 and Qwen 2.5, utilizing a randomly initialized MLP projector.

The Sapiens visual model, developed by Meta Reality Labs, focuses on handling human visual tasks, including 2D pose estimation, body part segmentation, depth estimation, and surface normal prediction. It has been trained on over 300 million human images, showcasing high-resolution image processing capabilities and excellent performance even in data-scarce conditions. Its straightforward design facilitates scalability, and its performance significantly improves with increased parameters, surpassing existing baseline models in multiple tests.

LongVA is a long context transformer model capable of processing over 2000 frames or 200K visual tokens. It achieves leading performance in Video-MME among 7B models. The model is tested on CUDA 11.8 and A100-SXM-80G and can be quickly deployed and used through the Hugging Face platform.

Florence-2, a high-performance visual foundation model developed by Microsoft, utilizes a prompt-based approach to handle a wide range of visual and vision-language tasks. The model can interpret simple text prompts to perform tasks like description, object detection, and segmentation. It is trained on the FLD-5B dataset, which consists of 540 million images with 5.4 billion annotations, mastering multi-task learning. Its sequence-to-sequence architecture enables strong performance in both zero-shot and fine-tuning settings, establishing it as a competitive visual foundation model.

Florence-2-large, developed by Microsoft, is an advanced vision foundation model that uses a prompt-based approach to handle a wide range of visual and visual-language tasks. The model can interpret simple text prompts to perform tasks such as image description, object detection, and segmentation. It is trained on the FLD-5B dataset, which contains 540 million images with 5.4 billion annotations, making it proficient in multi-task learning. Its sequence-to-sequence architecture enables it to perform well in both zero-shot and fine-tuning settings, proving to be a competitive vision foundation model.

llama3v is a state-of-the-art (SOTA) visual model based on Llama3 8B and siglip-so400m. It is an open-source VLLM (Visual Language Multi-Modal Learning Model) with model weights available on Huggingface, supporting fast local inference, and released inference code. This model combines image recognition and text generation by adding a projection layer to map image features to the LLaMA embedding space, enhancing its understanding of images.

Page Assist is a convenient web user interface that simplifies interaction with your local AI models. You can use it to interact with your local AI models within your browser, or as a web user interface for local AI model providers like Ollama Repo. Current features include sidebar task support, visual model support, a minimized local AI model web interface, internet search functionality, a sidebar PDF dialogue box, and document chat (PDF, CSV, TXT, MD formats).

VMamba is a visual state-space model that combines the advantages of convolutional neural networks (CNNs) and visual Transformers (ViTs), achieving linear complexity without sacrificing global perception. It introduces the Cross-Scan Module (CSM) to address the issue of direction sensitivity and can demonstrate excellent performance in various visual perception tasks. As the image resolution increases, it shows more significant advantages compared to existing benchmark models.

This paper introduces AIM, a family of visual models pre-trained using autoregressive objectives. Inspired by their textual counterparts, the large language models (LLMs), these models exhibit similar scaling properties. Specifically, we highlight two key findings: (1) the performance of visual features improves with increasing model capacity and dataset size, and (2) the value of the objective function correlates with model performance on downstream tasks. By pre-training a 70-billion parameter AIM on 2 billion images, we achieved 84.0% accuracy on ImageNet-1k using a frozen backbone. Interestingly, even at this scale, we observe no signs of performance saturation, suggesting that AIM may represent a new frontier in training large-scale visual models. AIM's pre-training is similar to that of LLMs and does not require any image-specific strategies to stabilize large-scale training.